report resumes 

ED 016 620 24 SE 

DIFFERENCES IN LEARNING OUTCOMES BETWEEN A NEW AND A 
TRADITIONAL CHEMISTRY COURSE. 

BY- MARKSt RONALD L. 

INDIANA STATE COLL., PA. 

REPORT NUMBER BR 5-8026 pyg DATE 

REPORT NUMBER CRP-S-317 
EDRS PRICE MF-$0.25 HC-$0.92 21P. 

DESCRIPTORS- #ACADEMIC ACHIEVEMENT, ^CHEMISTRY, ^COGNITIVE 
PROCESSES, ♦CRITICAL THINKING, ♦SECONDARY SCHOOL SCIENCE, 
♦TEST CONSTRUCTION, COGNITIVE ABILITY, EVALUATION, LEARNING, 
STUDENT CHARACTERISTICS, TESTS, ACHIEVEMENT TESTS, INDIANA 
STATE COLLEGE, PENNSYLVANIA, CHEMICAL BOND APPROACH, 

ASSESSED WERE COGNITIVE DIFFERENCES BETWEEN HIGH SCHOOL 
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PREFERENCES WERE CHOSEN FOR STUDY — (1) RECALL OF FACTS AND 
TERMS, (2) PRACTICAL APPLICATIONS, (3) CRITICAL QUESTIONING 
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Cl) CRITICAL QUESTIONING, AND (2) FUNDAMENTAL PRINCIPLE 
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DIFFERENCES BEING DUE TO (1) ABILITY, AND (2) ACHIEVEMENT 
WERE TESTED AND REJECTED. (DH) 
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I , Int reduction 



One of the many new curriciila developed for high school 
science courses recently is the Chemical Bond Approach (CBA) 
Proiect. Supported by private and National Science Foundation 
funds, the Chemical Bond Approach Project Committee was folded 
to develop the new course. Until the development of the CBA 
course, much of the chemistry being tau^t was based on memor- 
ization of specific chemical facts or terns and on practical 
application of these facts. It was felt by the CBk Ccmmit ee 
that for application of chemical knowledge to take place basic 
understandings of chemical theory had to be acqmred 
Thus, the CBA course of study revealed the importance of theory 
and experiment 0 To emphasize the theoretical approach the CM 
Committee chose, as a central theme for the course, the relation- 
ship between chemical bonds and the interactions of species in 
chemical systems. 

The develojment of the new course of study was one problem, 
but to evaluate the student’s fulfillment of the goals of the 
course was quite another. One study which was completed ^d 
which attempted to evaluate the new CBA cour^ 
achievement testing procedures ,(3) To the extent that the CBA 
and traditional courses are alike in goals, the study was suc- 
cessful, But by the nature of the two courses, the goals are 
considerably different. The problem was to find a measure which 
could reliably differentiate some of the aims and objectives of 
CBA chemistry as they were compared to traditional chemistry. 

During the 1961-1962 school year, Robert W„ Heath, of the 

Educational Testing Service, developed an _ 

called a Cognitive Preference Examinatio n, The examination was 
used to show how at least one of the goals of the new Physical 
Science Study Committee (PSSC) physics caffered from that of 
traditional physics. 



The problem of assessing student pirogress toward 
the distinctive goals of the new curricula is 
not a simple one. The body of psji'chometric skills 
now available has largely grown out of traditional 
educational practices. It seems necessaiy to a^ 
proach the problem from a frame of reference which 
is different in its conception of achievement and 
therefore different in method of measurement. 



The interest is not in whether the student can 
identify correct or incorrect information but 
rather in what is he likely to do with infoiraa— 
tion intellectually. If a goal of instruction 
is to change the student ’ s intellectual style 
;d.thin some academic subject, a test of such 
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’•achievement” must pemit time to demonstrate 
differing styles « The test items should allow 
the student to exhibit some preference in 
cognition o (4) 

Statement of the Problem s If , in fact , the CBA cxirriculum 
does have a goal of instruction which would change the intel- 
lectual style of the student (i.Co, the student's mode of pro- 
blem solving or inquiry) ^ a student; oriented in CBA chemistry 
should exhibit, cognitive preferences which are significantly 
different from those preferred by students enrolled in a tra- 
ditional chemistry course o The purpose of this study is to 
study the cognitive preferences of CBA and traditional chemistry 
students in an attempt to identify certain goal differences 
between the type of curricula „ In the fom of a question: is 
there a significant difference in cognitive preferences as ex- 
hibited by students taking the CBA chemistry course and students 
in the traditional course? 

Four cognitive preferences were chosen for the study; 

(1) memory or recall of specific facts or terms, (2) practical 
application, (3) critical questioning of information, and 
(4) identification of a fundamental principle,, Some other, or 
perhaps different, cognitive preferences are; (1) laboratory 
technique, (2) descriptive chemistry, (3) theoretical inter- 
pretation, (4) correlation of data with theoiy, etc. The 
former were chosen because of the close relation "memory” and 
"practical application” have to the older curriculum o "Critical 
questioning of chemical information" and "identification of a 
fundamental principle" should appeal to the CBA students. 

Sub-problems g 

1„ The cognitive preferences as exhibited by the students 
will be related to achievement in the two types of curriciilao 
In the form of a questions is there a significant difference 
in the four cognitive preferences chosen for the study as ex- 
hibited by students taking the CBA chemistry course and the 
students in the traditional course when related to achievement? 

2„ The cognitive preferences as exhibited by students 
will be related to their ability. In the form of a question; 
is there a significant difference in the four cognitive pref- 
erences chosen for the study as exhibited by students taking 
CBA chemistry course and the students in the traditional 
course when related to ability? 

Hypotheses g 

With a sample of classes enrolled in CBA chemistry and a 
sample of classes enrolled in traditionally taught chemistry. 
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the following hypotheses were tested; 

lo Students in the CBA chemistry course and the students 
in the control groups show no statistically significant difference 
in their preferences for fundamental principle and critical 
questioning as measured by the Cognitive Preference Test, 

2. Students in the CBA chemistry course and students in 
the control groups show no statistically significant difference 
in their preferences for memory of facts and practical applica- 
tion as measured by the Cognitive Preference Test, 

The hypotheses for the sub-problems are as follows; 

lo Students in the CBA chemistry course and students in 
the control groups show no statistically significant difference 
in their preferences for fundamental principle and critical 
questioning j, as measured by the Cognitive Preference Test, when 
these preferences are related to achievement test scores (Co- 
operative High School Test and CBA Final Examination) . 

2o Students in the CBA chemistry course and students in 
the control groups show no statistically significant differences 
in their preferences for memory of facts and for practical appli- 
cation, as measured by the Cognitive Preference Test, when these 
preferences are related to achievement test scores (Cooperative 
High School Test and CBA Final Examination) o 

3o Any statistical difference in cognitive preferences is 
related to ability (SCAT) , 

The above hypotheses will be accepted as significant at 
the O 0 O 5 level and highly significant at the 0,01 level of 
confidence , 
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II. Method 



The investigator developed in instmment to measure the 
cognitive preferences of the students in the CBA. and control 
groups. The instrument ( Cognitive Preference Test: High 

School Chemistry ) consists of 100 items, and gives the appear- 
ance of a four-option, multiple choice test. (See Appendix A). 
The introduction, or stem, provides the chemical information. 
Four ^answers” or preferences follow the item* Each of the 
four preferences is correct. The student was informed that 
each of the options was correct, and that he was to select the 
choice he preferred most in connection with the introductory 
information. 



Each of the items was designed so that the four options 
would designate a different form of cognitive preference in 
chemistry. The four preferences were stated above. The re- 
sults, obtained from the students who took the Cognitive Pref- 
erence Test , were scored as four separate subtests of 25 items 
each. The subtests corresponded to the four cognitive pref- 
erences . 



An item analysis was computed frcan. the Cognitive Preference 
Test . It was determined that each of the 100 items was a func- 
tional part of the test as a whole. Also, the reliability of 
the instrument was calculated and it was considered reliable for 
research piirposes. (Greater methodological detail can be found 
in Appendix B) . 

A list of all known CBA chemistry teachers was catagorized 
on a two-variable stratification table. The two stratification 
variables were (a) sex of the teacher and (b) size of community 
(over or under 100,000 population). A random sanqple of these 
teachers was selected and invited to participate in the study. 
The teachers that accepted the invitation were catagorized on 
the same two stratification variables used for the CBA popula- 
tion. A chi-square value of 0.31 indicated that the CBA group 
was a representative sample from the population. 



The non-CBA schools were selected. Invitations we^e issued 
to schools in proportion to the CBA population stratification. 
The distribution of the control sample was tested against the 
CBA sample by means of the chi-square test. No statistically 
significant difference was found. 



In addition to the Cognitive Preference Test , with its 
four subtests, the following tests were used in the testing 
program: (a) AoCoS o -N.S.T.A . Cooperat3.ve Examination ; High 

School Chtwni stry . Form 1961, a traditional achievement test, 

(b) The Chemical Bona Approach Pro.ject Final Examination , 

Part I, an achievement test for the new curriculum, and (c) The 
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School and College Ability Test , Parts I and IV , Form lA o 

The means and standard deviations of all tests, for both 
groups, were computed from the pre-test and post-test data. 

From these data, ”t” tests were computed to determine if there 
was a significant difference between CBA and control group 
means for the four cognitive preferences. The possibility that 
the differences in cognitive preferences could be accounted for 
by differences in ability was tested. 

Analysis of variance and intercorrelations of the mean 
scores of the seven variables (SCAT, CBA Final , AoCoS o Coopera - 
tive , Cognitive Preference Subtests) for both CBA and control 
groups were determined. This was in an attempt to investigate 
the sub-problem hypothesis that the cognitive preferences were 
related to achievement in the two types of curricula. 
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III. Results 



The me^ and standard deviation of all tests for the 
pre- est and post-test programs were computed for the (3BA 

S statistics presented in 

Tabj.e I are based i^jon cluster samples of 24 teachers of CM 

Md3?WhLr^? the ejqjerimental group (433 students) 
and 30 teachers of traditional chanistry who made up the con- 

.~1 group (622 students) for the post-test data. For 
the mean score for SCAT in Table I for the CM g^ufLITS 

o^^thr^AT^t arithmetic average of 24 CM cluster means ‘ 
on the SOkt test. Below that statistic the value of 5 29 

on^^rsCATf® deviation of the 24 CBA cluster means 



a cofflpilation of the data needed to test the 
mjor hypotheses to this study. The data necessary to test 

beca^°o?the is n°t shown inThis report 

because of the limitation on space and for the sake of claritv 

nf-Sr “ the next section, "Discussion,"^ 

data needed to test the hypotheses to the sub-problems. 
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TA6I£ I 



COHPABISON OF HEADS ON ALL FOST-TES1S 



(d.f.= 52) 





CBA 


CONTROL 






TEST STATISTICS 


GROUP 


GROUP 




SIGNIFICANCE 


(N=^4) 


(N=30) 


t 


LEVEL * 


Mean 


40.56 


37.58 


1.97 


0.06 


Standard 










SCAT Deviation 


5-29 


5.82 






Mean 


42.87 


44.05 


0.31 


n.So 


A«CoSo Standard 

Co po Deviation 


15o04 


13.05 


- 




Mean 


12.65 


9o33 


3.61 


OoOl 


CBA Standard 

Final Deviation 


3.69 


3o03 






Mean 


7.61 


8.99 


3ol4 


OoOl 


Cognitive; Standard 
Memory Deviation 


1.53 


1.69 






- ... Mean 


7o03 


6.25 


2.68 


0,01 


Cognitive; 


Critical Standard 
Quest o Deviation 


lol5 


1.03 






Mean 


4.53 


4.39 


Oo45 


n.So 


Cognitive; 

Pract 0 Standard 










Appo Deviation 


1.35 


0.94 






Mean 


7o09 


6o60 


2,01 


0.05 


Cognitive; 






Fundo Standard 

Prino Deviation 


0,88 


0,91 






* Values not reaching the 0o05 level are 


recorded as 


not 


significant (n.Se) unless otherwise indicated. 
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IV» Discussion 



The first major hypothesis was stated so that an attempt 
was made to determine if students in the CBA chemist ly course 
and students in the control groups showed any statistically 
significant difference in their preferences for fundamental 
principle and critical questioning as measured by the Cognitive 
Preference Test . The results of the analysis of the data for 
the post-tests are shown in Table I. 

The computed t-value for the difference between means of 
the CBA and control groups on the critical questioning and 
fundamental principle subtests were 2o68 and 2.01, respectively, 
With 52 degrees of freedom^, (24-1) + (30-1), the tabled t-value 
is 2o01 at the 0,05 level. Therefore, the mail hypothesis that 
there is no statistically significant difference between the 
CBA and control groiaps in their preference for critical ques- 
tioning and fundamental principle options to the test items 
can be rejected. 

The means for the CBA group and the control group on the 
critical questioning subtest are 7o03 and 6,25, respectively. 

The analysis would indicate that the differences between these 
means is highly sigiificant and that CBA students did have a 
stronger preference for the critical questioning option on the 
Cognitive Preference Test . Similarly, the higher mean for the 
CBA group on the fundamental principle subtest would indicate 
a stronger preference by that group for the fundamental prin- 
ciple option than the control group. 

The second major hypothesis related to an attempt to 
determine if students in the CBA group and students in the 
control group showed any statistically significant difference 
in their preferences for memory of facts and practical appli- 
cation as measured by the Cognitive Preference Test . 

It is shown in Table I that a t-value of 3ol4 was obtained 
for the difference between means for the memory subtest, and 
was highly significant beyond the one per cent level. For the 
differences between means on the practical application subtest, 
a t-value of 0,45 was obtained, which was not significant. The 
null hypothesis that the two methods of instruction produce no 
statistically significant difference in student preference for 
memory of facts was rejected. However, it could not be rejected 
for the practical application subtests , 

The results shown in Table I indicated that the students 
in the control group, with a mean of B,99> showed a stronger 
preference for the recall type option on the Cognitive Pref- 
erence Test than did the CBA group, as is indicated by their 
mean score of 7o6lo Further examination of Table VIII indicated 
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that the CBA and control groups had nearly identic^ pwferenc 
(a mean difference of 0 , 14 ) for the practical application option 

to the test items « 

Additional infoimation in Table I 
tin statistically significant difference between the CBA and con- 
?rof ^ups on the Cooneratlve Ex^a^. This ousted 

that the method of instruction did not significantly af fee 
student's facility on this conventional achievement test, ^h r, 
the table indicated that the two groups showed a statisticaUy 
si^ScLrdifference in their ability as ^as^d by ^e ^ 
and College Ability Test, Fom lA, Pm±s I M f was noted 

that a sieiifioant difference had appeared on this same test in 

the pre-test data. 

Also, Table I indicates that a highly significant difference 
existed between the means of the CBA and control groups on 
CBA Final vsysml nation . This difference showed an advartage for 
^ nBT ~group whiciTexhibited superior performance on the test 
designed for its own method of instruction. 

Analyses of variance were performed on 
if the ability of the two groups could account for 
ences in cognitive preferences. The results indicated that the 
cognit^e pSferences of these students could not be rela^d to 
their ability as measured by SCATo The hypothesis whic 
third sub-problem was rejected. 

When achievement test scores were related to the 
preferences of students by means of analysis of variance, it was 
found that neither the AoCoS o Cooperative Examnat^ 

CBA Final TCyamination were criteria from which 

cognitive preferences of these students. Thus, the h^o . , , 

thS^the cognitive preferences as expressed by students could be 

related to achievement were rejected. 
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Vo Conclusions a Implications and Re commendat ions 



The mean scores for the CBA group and the control group 
on the fundamental principle subtest were 7o09 and 6o60 respec- 
tively o While this difference is statistically significant at 
the O0O5 level of confidence^ it was felt by the investigator 
that a difference of approximately one-half an item on a 25-item 
test had little educational significance. That is, the obtained 
difference was not large enough to have curriculum revision 
implications o 

Means of 7o03 and 60 25 for the critical, questioning subtest 
were obtained for the CBA group and control group respectively 0 
It can be inferred that this difference has possible educational 
significance o For example, school administrators may wish to 
accept the CBA course because the student engages in inquiry 
rather than the memorization of facts. One may also infer that 
one of the major objectives of the CBA curriculum is being 
realized. The CBA course of study encourages the student to 
T ntfiypTet and analyze newly presented data from previously 

learned material. 



Statistical analyses revealed a highly significant differ- 
ence between the CBA group and the control group in their 
preference for the memory option in the Cognitive Preference 
Test items. This indicated that the control group had more of 
a preference for the memory or recall of information option 
than did the CBA group. It can be assumed that the traditional 
approach to teaching high school chemistry has as one of its 
goals of instruction the presentation of many facts and terms 
for the student to memorize. The CBA course has less emphasis 
on memorization. 

The practical application subtest data were very inter- 
esting in light of present day discussions in chemistry 
tion. The CBA course does not stress to any appreciable extent 
the practical application of chemical systems 5 yet, the CBA 
group and the control group had virt\ially eqiaal preference for 
that option in the test items. The interesting educational 
inference that can be drawn from these means is not so much 
their lack of a statistically significant difference, but their 
numerical magnitude. A low mean for the CBA group on the prac- 
tical application subtest could be assumed. However, if the 
traditional course has as much emphasis on practical application 
as it is purported to have, the students, at least the ones in 
this study, were not preferring it to the other goals of in- 
struction. 

Additional information of interest obtained from this 
study was that the CBA group and the control group showed no 
statistically significant difference on the A.CoSo Cooperative 
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^^^vATn^ nation . The educationally interesting observation that can 
be inferred from these data is the fact that the CBA group did 
as well on the traditional achievement test as did the groi^) 
tau^t in the traditional manner o The content of the A.CoSo' 
CooneratiVe Examin ation is such that very little of it could be ^ 
directly taught in the CBA course of study. However^ it appeared 
that the CBA students were able to accept related material and 
work with it in li^t of what they had learned in the CBA course » 
On the other hand^ the traditional students did not do nearly as 
well, on the 0^ Final Examination which stresses conceptualization 
as did the group of CBA students , 

It is apparent from the data presented above that the CBA 
course of study is achieving at least one of its goals of in- 
structiono The cognitive style of the students seem tied more 
to conceptual, learning rather than to a verbal structure. 

In light of the findings in this study and the conclusions 
drawn from them^ research in the following areas may be desirable; 

lo A study should be undertaken to determine if a two- 
option^ 100-item cognitive preference test measure the intellec- 
tual style of students in an experimental and control group with 
any greater predictability than was prepared in this study with 
the four-option, 100-item test., The two options chosen for t^s 
kind of study should be memory of facts and critical questioning. 

2o A study similar to this one should be mdertaken except 
that different cognitive preferences In the options to the test- 
items should be written « 

3o A study should be undertaken to determine if the labora- 
tory program in high school chemistry has any influence on the 
cognitive preferences of students. 

4o A study s imi lar to this one should be conducted but 
should compare cognitive preferences of students enrolj.ed in 
Biological. Sc,i.en.ce Curriculum Study biology with those enrolled 
in more tradl-tional courses o 

5„ A study shoul.d be undertaken to determine how much 
descriptive chemistry. In particular practical application to 
chemical systems, should be included in a hi^ school chemistry 

programo 

6o A replication of this study should be undertaken with 
the addition of a Hawthorne group to the CBA and control group. 
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VI. Summary 

This investigation ccmipared the cognitive preferences of 
students enrolled in Chemical Bond Approach Project (CBA) high 
school chemistry classes with the cognitive preferences of 
students enrolled in traditional high school chemistry classes. 

The comparison was made in an attempt to identiity certain goal 
differences between the types of curricula. 

The four cognitive preferences chosen for comparison in 
this study were: (a) memory or recall of specific facts, (b) 
practical application, (c) critical questioning of information, 
and (d) identification of a fundamental principle. 

Also investigated was the possibility that the cognitive 
pj>* 0 jf 0 jt 0 ];j(;es, as expressed by the students, could be related to 
achievement in the two types of curricula . The possibility 
that the students’ cognitive preferences were related to their 
ability was also investigated. 

The investigator developed an instrument ( Cognitive Pref- » 
erence Test; Hi^ School Chemistry ) to measin*e the cognitive^ 
preferences of the students in the CBA and control groups. The 
instrument consisted of 100 items, and gave the appearance of a 
four-option, multiple choice test. The introduction, or stem, 
provided the chemical information. The four ’’answers” which 
followed the stem designated a different form of cognitive pref- 
erence. The student was informed that each of the options was 
correct, and that he was to select the option he preferred most 
in connection with the introductory information. 

An item analysis was computed from the Cognitive Preference 
Test. It was determined that each of the 100 items was a func- 
tional part of the test as a whole. Also, the reliability of the 
instrument was calculated and it was considered i^liable for 
research puiposes. 

Based on the sex of the teacher and size of the community, 
a randcHi sample of twenty-six schools throughout the United 
States offering the CBA co\irse was representative of the CBA 
population, A random sample of thirty-two schools throughout 
the United States offering a traditional chemistry course con- 
stituted the control g3x>up, A chi-square test indicated the 
control sample was’ representative of the CBA sample when tested 
on the same stratification variables. 

In addition to the Cognitive Preference Test , the following 
‘tes'ts wer6 used in the testing progrsms (sj AoC«S o*-N#SoT^ 
Cooperative Examination : High School Chemistry a Fom 
(b) Th*=^ Ch emi cal Bond Approach Project Final Examination , Part I, 
and IcT ^ School and College Ability Test , Parts I and IV> 

Fom lA. 
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Analysis of the pre-test data indicated that there was no 
statistically significant difference between the CBA and control 
groups in their preferences for the memory of facts, practical 
application, fundamental principles or critical questioning options 
to the test items. 

The post-test analysis showed that a statistically significant 
difference did exist between the CBA and control groups in three of 
the four cognitive preferences. The CBA group exhibited a stronger 
preference for the critical questioning and fundamental principle 
options to the Cognitive Preference Test items . The control group 
showed a stronger preference for the memory of facts option. Anal- 
ysis of the data indicated that both groups had virtually the same 
preference for the practical application option. The mean score 
for each of the groups was significantly lower for the practical 
application option when compared to the mean scores for the other 
three cognitive preferences. 

An analysis of variance, factorial design, was computed to 
investigate the possibility that the cognitive preferences, as 
expressed by the students, could be related to achievement in the 
two curricula. One such relation which existed was that the CBA 
classes which had lower scores on the CBA Final Examination had a 
stronger preference for the practical application option than did 
the corresponding grovip of traditional classes. 

The possibility that the students* cognitive preferences were 
related to their ability was tested and discounted. 



13 




VII. References 



1. Garrett, Henry Statistics ^ Psychology Education 
New Yo](*k. Longjnans, Green and Co o , 1961 • P® 366 

2o puilford, J. Po Psychcanetric Methods o New Yorks McGraw- 
Hill Book Company. 1954. P» 528. 

3. Heath, Robert Wo ”A Study of Achievement in High School 

Chemistiy,'* Report the Chemical Bond Approach Project, 
Educational Testing Service » Princeton, New Jersey. 
October, 1962. 

4o Heath, Robert W» “Curriculum, Cognition, and Educational 

Measurement,” Education^ and Psychological Measurement, 
SSIV, NOo 2. Summer, 1964 o pp. 240-241. 



14 



VIII. Appendix A. 



The Cognitive Preference Test ; High School Chemistry . 



t- 



1 






[ 

I 

I 



► 

\ 



i 






A-1 





3X* Appendix B 

Methodological Detail 

The Coimitive Preference Test: Chemist ^ is a 100- 

item test. U copy of the test is the**tSt is that of a 

The general appearance of an item in the test is -onaTi o 

four option, multiple choice question. Each item has a 

p^enfs s^e l^oLation or data of a the 

?oiiT» notions is factually correct and related to the stOT of the 
L^.%he directions for the administration of the test read, in 

part, as follows: 

In this test you are to indicate which one of four 
choices you prefer. Each test item begins with ^ 
introductory statement or diagram. This mfomation 
is followed by four lettered choices. Each of these 

choices is correct . 

Read the introductoiy statement and aU four choices 
carefully. Select the choice you prefer most in 
connection with the introductory infoimation. Then 
blacken the corresponding space on your answer she t. 
Remember, all the infoimation given is factu^y 
correct. You should choose the answer that h|s ^2 — 
appeal or is most satisfying yoH because of the 
chemistry course you bad. 

lou may find that more than one choice for each test 
appeals to you. However, select the oj^ choice 
for each item. Be sure to answer every question, even 
though the decision may be difficult to make. 

The four options for each of the itens were written so that 
the reader could express his preference for a given style of 
thinking. To the^tation of the number of 

could e:q)ress a cogiitive preference which is most satisfying to 
him. 

indiwite a prartical applioatiw 
eiven in the stem. Another option designates the re^er s 
Sr^hSlenging or critically analystog the 
A fourth option has an appeal for a fundament^ ^ e?™ 

e:q)lains, at least in part, the information given in the stem. 

Item 87 from the test reads as follows; 
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Vegetable oils beccMe fats by combining the oil with 

hydrogen. 

(a) Crisco is the result of such a process. 

(b) The process is called hydrogenation. 

(c) Hydrogen can be added to certain unsaturated 
covalent bonds. 

(d) Addition of hydrogen to vegetable oils liberates 
heat. Sat\jrated fats are therefore more stable 
than unsat\irated fats. 

A s tudent may receive the most satisfaction by answering the 
above item with (a) . Such an answer may result as a score toward 
his "practical application" total score. A choice of (b) may count 
toward his "memory" total score. If the student prefers (c) as his 
answer, a point may be added to his "fundamental principle" score, 
(d) would be the correct answer for the "critical questioning" part 
of the test. 

Method of Evaluating the Results Obtained from the Cognitive 
Preference Test . Since each of the answers is correct, the Cogni- 
tive Preference Test would have an undesirable interdependent 
quality built into it. That is, if one answer were chosen by a 
student, three other correct answers would be eliminated frcM the 
total test score. For example, suppose a student were to have a 
preference of option (a) for an item while a second student were 
to have a preference for option (b) for the same item. The re- 
sult would be that the item would be evaluated one way for the 
one student while a different evaluation would be obtained for 
the second student. The resulting ipsative measurement for each 
student would be distributed about the mean of that student, not 
ibout the sample mean, "Individual differences in ipsative meas- 
urements have little meaning because there is not a sin^e scale 
for all individuals." (2) Therefore, the 1CX3 items were divided 
into four subtests corresponding to the four cognitive preferences. 
In this way, it was possible to evaluate, or grade, for the total 
score of just one of the cognitive preferences, A maximum score 
possible for any one of the four subtest was 25. 

The result of the division of the test into subtests was that 
four, 25-item tests were considered in the scoring of the test. 

So that each subtest would be evenly distributed throughout the 
test with respect to the other three subtests, the following 
assignments were made: The Memory Test was ccaaposed of test items 

1, 5, 9, 13 ... 97o Critical Questioning was scored for items 2, 

6, 10, 14 ... 98, The items designated for Practical Application 
were 3^ 7^ ^5 ... 99 « Items 4^ .o. 100 scored for 

the Fundamental Principle subtest. 

Validity of the Test Items in the Cognitive Preference Test . 
The validity of the items in the Cognitive Preference Test was 
determined by a jury of seven chemists. The jury members read 
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each of the test items to detennine (1) if the item was 
factually correct in subject matter, (2) if the cognitive 
preferences were truly related to the stem of the item, and 
(3) if the general content covered in the test was representa-* 
tive of a first course in chemistryo 

As a result of the comments and suggestions received from 
the chemists, several items were revised or rewritten o It was 
the consensus of the jury that the final form of the test was 
valid with inspect to the above points o 

Analysis of Each Item in the Cognitive Preference Test, 

An item analysis was performed to determine if each of the items 
in the Cognitive Preference Test was a functional part of the 
entire testo The procedure used was to identify the upper and 
lower twenty-seven per cent of the total score distribution for 
each subtesto ^ The proportion of examinees in the group repre- 
senting the highest twenty— seven per cent and lowest twenty— seven 
per cent provided the data to enter Flanagen’s table of noiroalized 
biserial coefficients as v Jjlished in Garrett (l) or the abac in 
Guilfordo (2) 

The obtained index estimated how well the item correlated 
with the total test score. In general, the higher the validity 
index the better the correlation, A validity index of 0.20 or 
higher is usually considered satisfactory although with more 
than 1000 examinees in tiiis study, a validity index as low as 
0,08 would be significant at the 0,05 level of confidence. 

The item analysis indicated that 97 of the items on the Cogni- 
tive Preference Test had a validity index of 0,20 or higher. 

The other three items !iad an index of between 0,13 and 0,20, 

Each item was considered to be a functional part of the test. 

R-*>liability coefficients of eac h of the subtests of the 
Cognitive Preference Test , Coefficients of reliability for 
each of the subtests of the Cognitive Preference Test were 
deteimined to furnish some estimate of the reproducibility 
of test scores. The results are shown in Table II, page 39. 

The procedure used to determine the coefficients was first 
to determine the Pearson product-moment coefficient of 
reliability between the odd and even items of each subtest. 

The Speaman-Brown fomula was then used to obtain an estimate 
of reliability of the whole test. It was concluded that the 
four suhtests were reliable and could be used for research 
purposes . 
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